ACTIONS OF VANISHING HOMOGENEITY RANK 
ON QUATERNIONIC-KAHLER PROJECTIVE SPACES 



LUCIO BEDULLI AND ANNA GORI 

Abstract. We classify isometric actions of compact Lie groups on 
quaternionic-Kahler projective spaces with vanishing homogeneity rank. We 
also show that they are not in general quaternion-coisotropic. 



1. Introduction 

Let M be a smooth manifold endowed with a smooth action of a compact Lie group 
G. We denote by c(G, M) the cohomogeneity of the action, i.e. the codimension of 
the principal orbits in M, and by H a principal isotropy subgroup. In [6, p. 194] 
Bredon proved the following inequality for the dimension of the fixed point set of 
a maximal torus T in G: 

dim M T < c(G, M) -vkG + rkH 

whenever M T is nonempty. Drawing on this fact, Puttmann introduced in [19] the 
homogeneity rank of (G, M) as the integer 

hrk(G, M) := xkG-xkH- c(G, M). 

In this paper we are interested in studying actions on quaternionic projective spaces 
and there are at least two reasons to consider actions with vanishing homogeneity 
rank. 

A first motivation comes from the following proposition which can be deduced from 



Proposition 1.1. Let M be a compact manifold with positive Euler characteristic 
acted on by a compact Lie group G. Then hrk(G, M) ^ 0. 

Indeed quaternionic projective spaces (and more generally positive quaternionic- 
Kahler manifolds, see [14]) have positive Euler characteristic, thus the actions we 
aim to classify are those with maximal homogeneity rank and this fact turns out 
to have remarkable consequences on the geometry of the action. 
Furthermore, in the symplectic framework, Hamiltonian actions with vanishing ho- 
mogeneity rank have a precise geometric meaning. Let the compact Lie group G act 
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on the symplectic manifold (M, u>) in a Hamiltonian fashion, then hrk(G, M) = if 
and only if every principal orbit O of the G-action is coisotropic, i.e. (T p O) w C T p O 
(such an action is said to be coisotropic). If further M is compact and admits a 
G-invariant w-compatible complex structure J, then (M, to, J) turns out to be a 
projective algebraic spherical variety, that is the Borel subgroup of G c has an open 
orbit in M (see [10]). Coisotropic actions on symplectic and Kahler manifolds have 
been extensively studied starting from [5] and have been classified on Hermitian 
symmetric spaces in |18) . [4] and [3]. Linear actions with vanishing homogeneity 
rank have been considered by several authors: the classification in the complex case 
can be deduced from [11] and [5] , (while absolutely irreducible real representations 
with vanishing homogeneity rank of compact Lie groups have been classified in 

m)- 

It is therefore rather natural to look for relations analogous to those found in the 
complex/symplectic framework in the quaternionic setting. 

Let M be a quaternionic Kahler manifold with positive scalar curvature and 
Z C End TM its twistor space. We say that a submanifold N of M is quaternion- 
coisotropic if for every p € N and J £ Z p we have J(T p A r )- L C T p N . The rea- 
son to consider the previous definition is the fact that the principal orbits of 
polar actions on compact symmetric quatcrnionic-Kahler manifolds are indeed 
quaternion-coisotropic [21] (in the same way as polar actions on Kahler manifolds 
have coisotropic principal orbits [17] ). 

A first result of this paper is an example of an action on the quaternionic projective 
space with vanishing homogeneity rank which is not quaternion-cosotropic (Exam- 
ple [3Tl]). We further determine in our main theorem all the compact Lie subgroups 
of Sp(n) acting with vanishing homogeneity rank on the quaternionic projective 
space in the following 

Theorem 1.2. Let p : G — > Sp(y) be a n- dimensional quaternionic representation 
of a compact connected Lie group. Then p induces a minimal vanishing homogeneity 
rank action of G on Pn(V r ) ~ IHIP"^ 1 if and only if one of the following is satisfied: 

(1) G = S^l)"- 1 and p = p s © . . . © p s © 1, where p s : Sp(l) -> Sp(H) is the 
standard representation and 1 is the trivial representation on H; 

(2) G = H x Sp(l) r and p = a © p s © . . . © p s , where a : H -> Sp(W) is one 
of the following 4(n — r) -dimensional quaternionic representation: 

(a) H = S(XJ{k) x U(n - r - k)) C SU(n - r) C Sp(n - r) and k is odd; 

(b) H = 5(U(l)Sp(fe)xU(n-r-A;)) C 5(U(2fc)xU(n-r-2fe)) c Sp(n-r); 

(c) H x Sp(l) rx VT^hIH is orbit equivalent to the isotropy representation 
of a quaternionic- Kahler symmetric space; 

(d) H = S P in(7) <g) Sp(l) C SO(8) © Sp(l) C Sp(8). 
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Note that many of these actions turn out to be non polar. 

The paper is organized as follows. In Section [2] we prove several lemmas about the 
homogeneity rank necessary for the proof of the main theorem. Results about polar 
actions on Wolf spaces and an example of a vanishing homogeneity rank action 
which is not quaternion-coisotropic are provided in Section [3l while Section |4] is 
devoted to the classification actions on HP™ -1 with vanishing homogeneity rank. 
Finally in the appendix one can find some tables we refer to in the course of the 
classification. Most of them are taken from [12] . 

2. Homogeneity rank of compact Lie group actions 

In this section we are going to prove several results about the homogeneity rank 
which will be useful in the classification of actions with vanishing homogeneity 
rank on quaternionic projective spaces. On the other hand these statements have 
an autonomous interest since they hold in general for actions of compact Lie groups. 
The following lemma allows us to by-pass (sometimes) the computation of the 
principal isotropy subgroup. 

Lemma 2.1. Let G be a compact connected Lie group acting on a compact manifold 
M . Take p e M and denote by 5 the difference rkG — rkG p and by E the slice 
representation at p. Then hrk(G, M) = hrk(G p , E) + 8. In particular if the G-orbit 
through p has positive Euler characteristic, then the action of G on M has vanishing 
homogeneity rank if and only if the slice representation at p does. 

Proof. Since the action of G on M is proper, it is known that at every point the 
slice representation has the same cohomogeneity as that of the action of G on M. 
Let E be the slice for the action at p. Let q £ E be principal both for the G-action 
on M and the G p -action on E (which is equivalent to the slice representation). 
Obviously (G p ) q = G q = G prin c- Thus 

hrk(G, E) = rk G p - rk (G p ) q - c(G p , E) 

= rk G - 5 - rk G q - c(G, M) = hrk(G, M) - 6 , 

and the conclusion follows. The last statement is a consequence of the well known 
fact that the homogeneous space G/G p has positive Euler characteristic if and only 
if rkG = rkG p . □ 

The following is an obvious but important property of the homogeneity rank which 
is a consequence of Proposition 2]. 

Lemma 2.2. Let p{ \ G — > GL(V^) (i — 1,2) be two finite- dimensional representa- 
tions of the compact Lie group G. Then hrk(G, V\ © V2) ^ hrk(G, Vi) + hrk(G, V2). 

Proof. Let t>, be a principal point of (G, Vi) for i = 1, 2. Denote by Oi = G/Hi the 
corresponding orbits. 
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Now consider the action of G on V\ ® Vi . The slice representation at (vi , 0) is 
V2©£/ where (G, V2) is the original action and U is a trivial G- module of dimension 
c(G, Vi). Now (t>2,0) is obviously principal for the slice representation, so that 
a principal isotropy subgroup H of (G,V\ © V2) is (Hi, V2) P rinc- Thus we have 
c(G, V) = c(G, Vi) + c(ifi, y 2 ) and therefore 

hrk(G,V) = rkG-rkH - c(G,V) 

= rkG - rk (#!, 1/ 2 ) princ - c(H u V 2 ) - c(G, V x ) 

= rk G - rk H x +hxk(H u V 2 ) - c(G, Fx) 

= hrk(G, Fx) + hrk(^i, F 2 ) < hrk(G, Vi) + hrk(G, Va) 

□ 

Another important tool in the classification carried out in Section |4] will be the 
following proposition which generalizes, in the case of positive Euler characteristic, 
the Restriction Lemma given in 10J for complex G-stable orbits of Hamiltonian 
isometric actions on compact Kahler manifolds. 

Proposition 2.3. Let G be a compact connected Lie group acting by isometries on 
a compact Riemannian manifold M . Let Y be a compact G-stable submanifold of 
M such that x(Y) > 0. I/hrk(G,M) = 7 then hrk(G,F) = 0. 

Proof. Let VmY be the normal bundle to Y in M . Since Y is compact, we can 
use the invariant version of the tubular neighborhood theorem (see e.g. [51 p. 306]) 
to get a G-equivariant diffeomorphism of an open G-invariant neighborhood U of 
the zero section of vmY onto an open G-invariant neighborhood W of Y in M. 
Now, since W is open in M and G acts with vanishing homogeneity rank on M, 
the G-action on W has vanishing homogeneity rank too, hence also hrk(G, U) = 0. 
Consider now the restriction of the natural projection ir\ u : U — ► Y. Let y € Y be 
such that 

(1) y is principal for the action of G on Y; 

(2) F := 7rr (y) C U has non-empty intersection with M pr i nc . 

Now consider the action of G y on F and take x £ F such that 

(1) x 6 Af princ ; 

(2) x is principal for the action of G y on F. 

Since the action of G y on F = VM(Y)y 1S linear, the homogeneity rank of this action 
is non-positive ( [T9J, corollary 1.2]), i.e. 

dim F dim G„ — dim G x + rk G v — rk G x ■ 
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Thus we can compute 

c(G, Y) = dim Y — dim G + dim G y = dim X — dim F — dim G + dim G y 

^ dim X — (dim G y — dim G^ + rk G y — rk G^ ) — dim G + dim G y 

= c(G,X)-rkG y +rkG x =rkG-rkG y , 

so that hrk(G, Y) 0. On the other hand the positive Euler characteristic of Y 
obstructs actions with positive homogeneity rank (Proposition [TTT]) and the claim 
follows. □ 

In the case of the quaternionic projective space we deduce also the following useful 
consequence 

Corollary 2.4. Let G± and G2 be closed subgroups respectively of Sp(ni) and 
Sp(n 2 ). Assume that the action of G = Gi x G 2 on P H (H ni ©H™ 2 ) ~ MF ni+n2 - 1 
is 3-coisotropic. Then Gi acts 3-coisotropically on Pjj(H ni ) ~ HP™* . 

Proof. Simply take two non-zero vectors v\ and v 2 respectively in H™ 1 and HP 2 and 
consider the orbits Oi — G- [vi] ~ HP™ i_1 . Now apply Proposition ^. 31 to the orbits 
O x and 2 . □ 

3. QUATERNION-COISOTROPIC ACTIONS AND THE VANISHING OF HOMOGENEITY 

RANK 

In order to introduce the right notion of "coisotropic" actions in the quaternionic 
setting, it is necessary to fix some notation. Let (M, (?) be a Riemannian manifold 
and V its Levi-Civita connection. A quaternionic-Kahler structure on M is a V- 
parallel rank 3 subbundlc Q of EndTAf, which is locally generated by a triple of 
locally defined anticommuting g-orthogonal almost complex structures ( Ji, J 2 , J3 = 
Ji J 2 ). Recall that a quaternionic-Kahler manifold is automatically Einstein, hence 
if its scalar curvature is positive it is automatically compact. Here we consider only 
positive quaternionic-Kahler manifolds. 

A submanifold N of M will be called quaternion- coisotropic if for every p 6 N and 
J £ Q p we have J(T p A) J - C T p N. Trying to seek the analogy with the symplectic 
context, it is rather natural to consider the following situation. 

Definition 3.1. Let (M,g,Q) be a quaternionic-Kahler manifold. We say that the 
action of a compact Lie group of isometries of M is quaternion-coisotropic if the 
principal orbits are quaternion-coisotropic submanifolds of M. 

Recall that an isometric action of a compact Lie group G on a Riemannian manifold 
M is said to be polar if there is an embedded submanifold S (a section) which meets 
all principal orbits orthogonally. In [21] it is proved, using the classification results 
of [IT] and [13] , that quaternion-coisotropic actions generalize polar actions on Wolf 
spaces [211 Theorem 4.10] in the same manner as coisotropic actions generalize polar 
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actions on compact Kahler manifolds ([IE])- The classification of polar actions on 
quaternion projective space has been obtained by Podesta and Thorbergsson. Here 
we restate the classification theorem because in the statement of [T7] a (trivial) case 
is missing. 

Theorem 3.2. [17j The isometric action of a compact Lie group G on HP"^ 1 is 
polar if and only if it is orbit equivalent to the action induced by a n-dimensional 
quaternionic representation pi ® ■ ■ ■ ® Pk where pi is the isotropy representation of 
a quaternionic- Kahler symmetric space of rank one for i = 1, . . . , k — 1 and pk is 
one of the following: 

(1) the isotropy representation of a quaternionic- Kahler symmetric space of 
arbitrary rank; 

(2) the trivial representation on a 1-dimensional quaternionic module H. 

Note that the missing case (this including a trivial module) is easily seen to be 
quaternion-coisotropic. 

In spite of these analogies, the parallel with the symplectic setting does not go 
further, indeed we have the following 

Example 3.1. Consider the action of G = U(/c) x XJ(n — k) C U(n) C Sp(n) 
on M = HP" -1 . It is not hard to see that, for k ^ 3, the Lie algebra of principal 
isotropy subgroup is isomorphic to u(fc — 2)©u(n — k — 2) whence the cohomogeneity 
of the action is 4 and the homogeneity rank vanishes. Suppose now that the principal 
orbits are quaternion-coisotropic and consider the lifted action of G on the twistor 
space Z = CP 2 ™ -1 . In general, when we lift an isometric action with hrk = of 
a compact Lie group on a positive quaternionic-Kahler manifold, three cases may 
occur according to the cohomogeneity of the action of a principal isotropy subgroup 
G p on the twistor line Z p ~ CP 1 : 

(1) The action of G p on Z p is transitive. In this case c(G,Z) = c(G,M). Fur- 
thermore a G-principal orbit of Z is G/G z . If we take into account the 
homogeneous fibration G/G z — > G/G p where G p /G z = S 2 and the fact 
that S 2 has positive Euler characteristic, we have rkG z = rkG p , and we 
can compute 

hrk(G,Z) = rk G - rk G z ~c{G,Z) 

= rk G - rk G p -c(G,M) =0 

(2) The action of G p on Z p has cohomogeneity one. Again, if z is principal for 
the Gp-action on Z pi then it is principal for the G-action on Z . Furthermore 
the homogeneous fibration G/G z — > G/G p has fibre S 1 = G p /G z , hence 
rkG z = rkGp — 1. Now, taking into account that c(G, Z) — c(G, M) + 1, 
by a dimensional computation we obtain that also in this case the G-action 
on Z is coisotropic. 



3-COISOTROPIC ACTIONS 



7 



(3) The connected component of the identity of G p acts trivially on Z p . In this 
case the G-action on Z is no more coisotropic. 

One easily verifies that in our case the homogenity rank of the lifted action is 
—2, that is the G-action on Z is not coisotropic. This implies that the connected 
component H of G p acts trivially on the twistor line Z p ~ CP 1 . Now denote by 
v the normal space to the G-orbit at p which is acted on trivially by H . On the 
other hand, if Ji, J2, J3 are three generators of the algebra Q p , these are fixed by 
H thanks to the argument above. Thus H pointwisely fixes the three 4-dimensional 
mutually orthogonal subspaces J\v, Jiv and J3V of T p G ■ p. But this is impossible 
since we claim that a subspace of T p G ■ p fixed by H has dimension 8. Indeed in 
correspondence to a principal point we have the following reductive decomposition: 

u(k)®u(n-k) = u(/c-2)©u(n-fc-2)©u(2)©u(2)©(C 2 (g)C fe ~ 2 )©(C 2 ®C™-' £ - 2 ). 

Hence we can identify the tangent space to the principal orbit with 

u(2) © u(2) © (C 2 <g> C fe " 2 ) © (C 2 © C n - k - 2 ) 

on which u(fc — 2) © u(n — k — 2) acts. Then H fixes the two copies of u(2), thus 
has dimension 8, as claimed. 

4. Actions with vanishing homogeneity rank on HUP" -1 : proof of 

Theorem 11.21 

The entire section is devoted to prove Theorem 11.21 In order to achieve the classi- 
fication, the following remark will be useful. Let G be a compact Lie group acting 
by isometries on a compact quaternionic-Kahler manifold M. If G' is a closed sub- 
group of G acting on M with hrk(G', M) = 0, the same is true for G. Indeed every 
compact quaternionic-Kahler manifold has positive Euler characteristic (see [14j 
Theorem 0.3]). As already observed this forces the homogeneity rank to be non- 
positive. But hrk(G',M) < hrk(G, M), by [f, Proposition 2]. Thus it is natural to 
say that a G-action with vanishing homogeneity rank on a manifold M is minimal 
if no closed subgroup G' of G acts on M with vanishing homogeneity rank. 
From now on we fix M = HP™ -1 — Sp(n)/Sp(l)Sp(n — 1). Since the identity com- 
ponent of Iso(HP™ _1 ,5) is Sp(n), we go through all the closed subgroups of it, 
starting from the maximal ones and then analysing only the subgroups of those 
giving rise to vanishing homogeneity rank actions. 

We proceed, in some sense, by strata: the first level is made by the maximal con- 
nected subgroups of Sp(n) listed in Table [3] in the appendix, then we pass to the 
maximal connected subgroups of the groups of the previous level and so on. 
Before starting the classification we make two remarks: 

(1) Cohomogeneity one G-actions on M (with positive Euler characteristic) 
have automatically hrk(G, M) = 0. Indeed in this case rkG — rkG pr i, lc ^ 1 
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and cannot be zero since otherwise the homogeneity rank would be odd 
which is impossible since M has even dimension. 
(2) A necessary dimensional condition for an action of G on M to have vanish- 
ing homogeneity rank is that 

(4.1) dim G + rk G ^ dim M. 

Finally in order to clarify our procedure we make one more observation. When 
considering the irreducible representations of simple Lie groups one must often check 
the dimensional condition (|4.ip or a variation of it. This is made easier by the fact 
that if (ci, . . . , c„) are the coefficients of the maximal weights of the representation 
of a rank n simple Lie group, then the function 

(ci, . ..,c„,) h-> deg(p (cii ... iCn) ), 

is strictly monotonic, i.e. if p and pf are two irreducible representations of a sim- 
ple compact Lie group with highest weights A and A', given by (ci, . . . , c„) and 
(c 1; . . . , c' n ) respectively, and if Cj ^ d i for all % and ci < c\ for at least one i, then 
deg/3 < degp' (see [16]). Then, in many cases it is sufficient to test the dimensional 
condition for the fundamental representations, and go further only if the condition 
is satisfied. 

4.1. Maximal subgroups of Sp(n). 

4.1.1. G = U(n). The action of U(n) on HP™ _1 , has cohomogeneity 1, thus has 
vanishing homogeneity rank. 

4.1.2. G — Sp(fc) x Sp(n — k) (1 ^ k ^ n). These subgroups act by cohomogeneity 
1 on HP"" 1 , thus hrk = 0. 

4.1.3. G — SO(p) <g> Sp(g) (n = pq, p ^ 3, q ^ I). For q ^ 2 we can compute 
the slice representation at the quaternionic line I spanned by a pure element of 
W (g> R iq . The algebra of the stabilizer is o(p - 1) ©Sp(l) ® Sp(q - 1) acting on 

(4.2) W- 1 ® {U ®R i{q ' iy ) 

where U can be seen as the 3-dimensional vector space of the imaginary quaternions 
on which Sp(l) acts by conjugation (see [12l p. 590]). Note that Sp(l) acts also on 
jj4(q-i) ^ jgjg-i |-,y r ight multiplication. If p is odd the G-orbit of £ has positive Euler 
characteristic so we can easily rule out this case by observing that the irreducible 
factor R p_1 (giH 9-1 , regarded as a complex representation, does not appear in Kac's 
list pi]. 

So we are left to consider the cases in which p is even. To get rid of the action 
on the slice of the unitary quaternions, let us consider the stabilizer of a principal 
element of R p_1 (g> U: such an element is of the form v\ ® i + V2 <E> j + (g> k, 
where vi,V2,V3 are linear independent elements of M p_1 . Here the algebra of the 
stabilizer H is o(p — 4) © sp(q — 1) and the slice contains as a direct summand the 
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tensor product of the standard representations V = M p ~ 4 <g> M 4 ' 9-1 ). Since at this 
level <5 = rkG — rkii = 3, applying Lemma \2.1\ in order to exclude also this case 
it is enough to show that hrk(if, V) ^ —4. Indeed this is easy to verify once we 
subdivide into three more subcases and we compute explicitly the principal isotropy 
of H on V. If q ^ p — 4 then (j pr inc — 5p(q — p + 3); if p — 6 ^ q ^ p — 3 then f) pr i nc is 
trivial; if q ^ p — 8 then f) pr inc = sp(p — q — 6) (see e.g. [9j p. 202]) and in all these 
cases hrk(if, V) ^ —4 (note that the equality holds only if q = 1). 
For g = 1 this is the action on HP P_1 induced by the isotropy representation of the 
quaternionic-Kaehler symmetric space SO(p + 4)/SO(p) x SO(4), thus it is polar 
by Theorem 13.21 To determine whether it has vanishing homogeneity rank or not 
we have to distinguish according to the parity of p. If p is odd the G-orbit of £ 
has positive Euler characteristic, the slice representation at £ is real and appears in 
the list of [7] since it is orbit equivalent to the isotropy representation of the real 
Grassmannian of 3-planes in M. p+2 . Thus it has vanishing homogeneity rank. When 
p is even, at the first step the slice is given by R p_1 ® U and with easy computations 
we find that the principal isotropy is f) pr mc = sp(p — 4), c = 3 hence hrk = 0. 

4.1.4. G = p(H) with p complex irreducible representation of quaternionic type of 
the simple Lie group H. In this case the dimensional condition that should be 
satisfied becomes dim if + rkH > 2degp — 4. Going through all the representations 
of this type, and using the argument referred to at the beginning of Section [H the 
following cases remain: 

(1) the representation of SU(6) on A 3 C 6 ; 

(2) the representation of Sp(3) with maximal weight (0,0, 1); 

(3) the spin representation of Spin(ll); 

(4) the two half-spin representations of Spin(12); 

(5) the standard representation of E7 on C 56 . 

(6) the standard representation of SU(2); 

Except for SU(2), that gives rise to a homogeneity rank zero action, since it has 
cohomogeneity one on HP 1 ~ S 4 , the other cases can be treated using the fact 
that all of them admit a totally complex orbit (see [2] and also PQ). These totally 
complex submanifolds arc Hermitian symmetric spaces, and therefore have positive 
Euler characteristic. Thus we compute the slice representation on these orbits, 
obtaining: 

(1) SU(3) x SU(3) ■ U(l) on C 3 <g> C 3 <g> C; 

(2) SU(3)-U(1) onS' 2 (C 3 )®C; 

(3) SU(5)-U(1) on A 2 (C 5 )®C; 

(4) SU(6)-U(1) on A 2 (C 6 )(g)C; 

(5) E 6 ■ U(l) on C 27 ® C. 
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These all give rise to vanishing homogeneity rank actions, since they are all multi- 
plicity free pTj . 

Let us remark that all of these actions on the quaternionic projective space are 
polar. 

4.2. The subgroups of U(n) C Sp(n). First note that the maximal compact 
connected subgroups of U(n) are SU(n) and those of the form Z ■ H where Z is 
the center of U(n) and H is a maximal compact connected subgroup of SU(n) (see 
Tabled]). 

Certainly SU(n) has vanishing homogeneity rank on HP™ -1 since it has the same 
orbits of U(rt), so let us go through the remaining cases. 

4.2.1. G = Z ■ S(U(fe) x U(n - k)) = U(fe) x U(n - k). We start by computing 
the slice representation at the class of the identity in Sp(n)/Sp(l)Sp(n — 1). The 
stabilizer is given by the intersection of G with Sp(l)Sp(n — 1). In this way we get 
U(l) x U(fc — 1) x U(n — k) acting on the slice 

£ = (C* <g> (C' 0-1 )*) © (C* <8> (C n-fe )*) © (C* <8 C n - k ) . 

Now it is immediate to see that the principal isotropy group is isomorphic to U(fc — 
2) x U(n — k — 2) so that the cohomogeneity is 4 and the action has vanishing 
homogeneity rank. 

Remark 1. Observe that the slice representation we just considered is complex, in- 
decomposable and has vanishing homogeneity rank, though it does not appear in the 
classification of Benson and Ratcliff |5j . In fact they consider only representations 
(G, V) which are indecomposable for the semisimple part of G. 

4.2.2. G = Z ■ Sp(fc) with n — 2k. Proceeding as before we determine the orbit 
through the class of the identity in Sp(n)/Sp(l)Sp(n — 1). Again we get an orbit 
with positive Euler characteristic, more precisely the Lie algebra of the isotropy 
is 3 © u(l) © sp(fc — 1) and the slice representation is given by H fc_1 © C , where 
the 1-dimensional factor 3 acts (non-trivially) only on C and u(l) acts by scalar 
multiplication on H fe_1 . Thus the algebra of the principal isotropy is isomorphic to 
u(l) © sp(k - 2) and hrk(G, HP" -1 ) = 0. 

Note that the action of the center here is essential: Once the action of Z is removed, 
there is a trivial module in the slice representation. Therefore Sp(fc) C SU(2fc) C 
Sp(2fc) does not have hrk = on HP" -1 . 

4.2.3. G = Z ■ SO(n). First consider the totally complex orbit of U(n) 3 SO(n) 
which is CP" -1 canonically embedded in HP" -1 . This orbit in its turn contains a 
Lagrangian G-orbit (RP n_1 canonically embedded). Here the 1-dimensional factor 
of the isotropy 3 © o(n — 1) acts on the slice IR" -1 © C 2 ^" -1 ) © C* only on the second 
module. From this one easily sees that g pr i nc ~ o(n — 4) and the cohomogeneity is 
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therefore 5. Thus hrk(G, HP" 1 ) = —2 and the action has non-zero homogeneity 
rank. 

4.2.4. G = Z ■ SU(p) <g> SU(g) (n = pq and p,q^2). Here G acts on P H (C P ® C q © 
(C p ® C 9 )*). The orbit through the quaternionic line spanned by a pure element 
of C p © C 9 is the product of two complex projective spaces CP P_1 x CP 9-1 and 
therefore has positive Euler characteristic. So we are in a position to apply the 
criterion deriving from Lemma 12.11 The slice representation contains the module 
CP- 1 (g>C 9 - 1 e(C 3 '- 1 <g)C 9 - 1 )* on which3©u(l)©u(p-l)©u(l)©u(g-l) acts. If 
p ^ 3 this module does not appear in the classification of [5] , thus the corresponding 
action has non-zero homogeneity rank. The case p — 2 is left to consider: If q 5 
the dimensional condition (|4.ip is not even satisfied, if q ^ 6 it is easy to find 
directly that the principal isotropy is su(q — 4), so that the homogeneity rank is 
-2. 

4.2.5. G = Z ■ p(H) with p irreducible representation of complex type of the sim- 
ple Lie group H . If G acts with vanishing homogeneity rank on HP 71-1 then, by 
Proposition ^. 31 it acts coisotropically on the G- invariant totally complex subman- 
ifold L = CP"" 1 = U(n)/U(l) x U(n - 1) and, since Z acts trivially on L this is 
in turn equivalent to the fact that the representation of p{H) c x C* on C" is mul- 
tiplicity free. Using Kac's list [11], and taking only the representations of complex 
type we get the standard representation of SU(n), the representations of SU(n) 
on A 2 (C") with n ^ 5 and on S^C"), the half-spin representation of Spin(lO), 
the standard representation of Eq on C 26 . We have to consider those representa- 
tions of complex type satisfying the dimensional condition that in this case becomes 
dim H + rk H ^ 4 deg p — 6. The only remaining case is the first one and has already 
been treated in subsection 14.21 

4.3. The subgroups of U(fc) x U(n — fc) C U(n). Except the diagonal subgroup 
(when 2k — n), the maximal compact connected subgroups of U(fc) x U(n — fc) are 
S(U(k) x U(n — fc)) and those of the form H x U(rt — fc) where H is a maximal 
compact connected subgroup of U(fc). For the subgroups of this form we can apply 
Corollary |2. 41 arguing that H must necessarily act with vanishing homogeneity rank 
on HP fe_1 . Thus for H we have only two possibilities: either H = U(fci) x U(fc2) 
(with fci + fc2 = fc) or H = Z ■ Sp(fc/2) (when fc is even). 

4.3.1. H — U(fci) x U(fc2). We can exploit the previous computations and consider 
the orbit CP fel_1 C CP fe " 1 C CP"" 1 C HP" -1 ; so the slice representation is given 

by 

C* © ((C fel-1 )* © C fc2 © (C fc2 )* © C n ~ k © (C"- fc )*) . 

on which U(l) x U(fci — 1) x U(fc2) x U(n — fc) acts. Analogously to a previous case it 
is easy to see that the principal isotropy group is isomorphic to U(fci — 2) x U(fc2 — 
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2) x U(n — k — 2) so that the cohomogeneity is 8 and the action has homogeneity 
rank equal to —2. 

4.3.2. H = Z ■ Sp(fc/2). We can compute the slice representation at the class of 
the identity in Sp(n)/Sp(l)Sp(n - 1). The intersection of g with sp(l) © sp(n - 1) 
is u(l) © u(l) © sp(k/2 - 1) © u(n - k) acting on the slice 

C n - k © (C n - fe )* © H^ 2 " 1 © C , 

where one of the two 1-dimensional copies of u(l) acts on every module and the other 
only on the first two modules. Now it is immediate to see that the principal isotropy 
subalgebra is isomorphic to sp(k/2 — 2) © u(n — k — 2) so that the cohomogeneity 
is 5 and the action has vanishing homogeneity rank. 

4.3.3. G = U(fc) A C U(fc) x U(fc) with n = 2k. In order to conclude that U(fc) A 
has non-zero homogeneity rank on HP™ -1 it is sufficient to observe that U(A;)a C 
Sp(/c)a C Sp(fc) x Sp(fc) and that the action of Sp(fe)A on HP™ -1 is equivalent 
to that of Sp(fc) C U(2k) since the standard representation of Sp(fe) on C 2k is 
self-dual. 

4.4. The subgroups of G = Z(U(n)) ■ Sp(k) C U(n) (with n = 2k). Now we are 

going to show that the action of Z(\J(n)) ■ Sp(fc) C U(n) is minimal as vanishing 
homogeneity rank action. The maximal compact connected subgroups of G other 
than Sp(fc) (that we have considered in a previous step) are of the form Z ■ H where 
H is a maximal compact connected subgroup of Sp(fc). 

4.4.1. H = U(k). As for this subgroup the conclusion follows immediately from 
the observation that Z ■ U(k) is contained in Z ■ SO(2k) which does not act with 
vanishing homogeneity rank on HP™ -1 . 

4.4.2. H — SO(p) (8 Sp(g) with 2pq — n. If Z ■ H acts with vanishing homogeneity 
rank on HP™ -1 , then it should act coisotropically on the totally complex \j(2pq)- 
orbit CP 2p9_1 , but this is not the case as one can deduce from the list of [11] and 
®- 

4.4.3. H = p(H') C Sp(fc) where p is an irreducible representation of quaternionic 
type of the simple Lie group H' . We can argue as in subsection 14.2.51 that is we 
apply our version of restriction lemma combined with the classification of Kac's. In 
this way we find no proper subgroup H of Sp(/c). 

4.4.4. H — Sp(r) x Sp(fc — r) with 1 ^ r ^ k — 1. Here it is sufficient to note that 
Z{U{2k)) -His a subgroup of Z(U(r)) • Sp(r) x Z(U(fc - r)) ■ Sp(fe - r) whose action 
on HP" -1 has non-zero homogeneity rank. 



3-COISOTROPIC ACTIONS 



13 



4.5. The subgroups of Z(U(k)) ■ Sp(r) x U(n - k) C U(n) with fc = 2r. Now we 
prove that the vanishing homogeneity rank action of Z(U(2r)) ■ Sp(r) x U(n — 2r) is 
minimal. Since the action of Z(U(2r))Sp(r) is minimal, by Proposition ^. 31 the only 
subgroups we need to consider are of the form Z(U(2r)) ■ Sp(r) x H, where H is a 
maximal compact connected subgroup of U(n— 2r) acting with vanishing homogene- 
ity rank on EDP™ _2r_1 . There are three possibilities for H: Hi = U(fci) x U(k 2 ) with 
k 1 + k 2 = n- 2r, H 2 = Z(U(n - 2r)) • Sp(^^) (when n is even), H 3 = SU(n - 2r). 
The subgroup Z(U(2r)) • Sp(r) x i2i is contained in U(2r) x U(fci) x U(fc 2 ), hence 
its action has non-zero homogeneity rank. 

The subgroup Z(U(2rj) ■ Sp(r) x iJ 2 need to be treated explicitly, finding the in- 
tersection of it with Sp(l)Sp(n — 1). In this way we get the isotropy subalgebra 
[ = u(l) © u(l) © u(l) © sp{r - 1) © sp(n/2 - r) acting on the slice 

HT" 1 © W n/2 - r © W n/2 - r © C © C . 

Since the abelian subalgebra of [ acts on the 1-dimensional modules, this action 
has vanishing homogeneity rank on each irreducible submodule, nevertheless it is 
easy to see that the principal isotropy is sp(r — 2) © sp(n/2 — r — 2). Therefore the 
cohomogeneity is 8 and hrk(Z(U(2r)) ■ Sp(r) x H 2 M^ 1 ) = -2. 
As for Z(U (2r)) • Sp(r) x H3 it is sufficient to observe that it induces on the quater- 
nionic projective space the same action of Sp(r) x U(n — 2r), which has non-zero 
homogeneity rank. 

This concludes the analysis of the subgroups of U(n) C Sp(n). 

4.6. The subgroups of G = p{H) with p irreducible representation of 
quaternionic type of the simple Lie group H. We have to examine only 
those subgroups that in case 14.1. "41 give rise to vanishing homogeneity rank actions. 
We exclude all of them simply noting that none of the subgroups of maximal di- 
mension satisfy the dimensional condition (|4.1[1 . The list of subgroups of maximal 
dimension is given in [15] and can be found also in |12j . 

4.7. The subgroups of G = SO(n) <E) Sp(l). Now we prove that the action of 
SO(?i) © Sp(l) is minimal except for n = 8. 

A maximal compact connected subgroups of G is conjugate to one of the form 
Hi © H 2 where Hi is either a compact connected maximal subgroup of SO(n) or 
SO(n) itself, and H 2 is either Sp(l) or U(l). The subgroup SO(n) © U(l) is the 
same as Z(\J(n)) ■ SO(n) C U(n) that we have already excluded (see case 14.2.31 . so 
let us turn to the case Hi © Sp(l) and look at Table [T] for maximal subgroups of 
SO(n). 

4.7.1. Hi = U(fc) where n = 2k. It is easy to find the slice representation at the 
quaternionic line £ spanned by a pure element of IR fc © M 4 starting from (|4.2|) . The 
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stabilizer subalgebra is u(k — 1) ffisp(l) acting on 

C*- 1 ffi R E 3 ffil 3 

where R 3 stands for the adjoint representation of o(3) ~ sp(l). It follows imme- 
diately that the principal isotropy subalgebra is isomorphic to u(k — 4) if n ^ 5, 
otherwise it is trivial. In any case the homogeneity rank is -4. 

4.7.2. H Y = S(0(fc) x 0(n - k)). The isotropy subalgebra at I G HP"" 1 is o(k - 
1) o(n — k) ® sp(l) acting on 

R^ 1 2> R 3 © R n - k ffi R 3 ffi R n - k . 

Here, in the general case, we are not allowed to skip the computation of the principal 
isotropy subalgebra. Nevertheless it is not hard to find that it is isomorphic to 
o(k — 4) ffi o(n — k — 4) for k, n — k ^ 6 so that c — 13 and hrk = —8. If either k 
or n — k are smaller than 6, a similar argument leads to the same conclusion. The 
remaining low-dimensional cases can be excluded using (|4.1[) . 

4.7.3. iJj = SO(p) (8 SO(q) with n = pq. The isotropy subalgebra at £ € HP™ -1 is 
o(p - 1) ffi o(q - 1) ffisp(l) acting on 

S = (RP- 1 ® R 9_1 ) ffi (W- 1 ffi R 9_1 ® R 3 ) ffi (W- 1 <g> R 3 ) ffi (R 9_1 ffi R 3 ) . 

Let us distinguish three subcases according to the parity of p and q. If p and q 
are odd then the orbit through I has positive Euler characteristic but the real 
irreducible module R p_1 % R 9_1 ® R 3 has negative homogeneity rank (it does not 
appear in the classification of [7]). 

If only one among p and q is even (say p), then the orbit has no more positive 
Euler characteristic but, with the notations of Lemma |2. II we have 5=1. Thus it 
is sufficient to show that hrk(G£,S) ^ —2. Thanks to Lemma |2~21 

hrk(G £ ,S) s; hrk(0(p- 1) x 0(q - 1) x 0(3), R^ 1 ffiR^ 1 ffiR 3 ) + 
hrk(0(p - 1) x 0(3), R p_1 <g> R 3 ) < -2 . 
If both p and q are even, we have 6 = 2, but 

hrk(G £ ,S) < hrk(0(p- 1) x 0(q - 1) x 0(3), R^ 1 ffiR 9 " 1 ffiR 3 ) + 
hrk(0(p - 1) x 0(3), RP- 1 <g> R 3 ) + 
hrk(0(<7 - 1) x 0(3), R 9_1 ffi R 3 ) < -3 . 

4.7.4. Hi = Sp(p) ffi Sp(g) wifft n = 4pg ^ 8. This action has no orbit of positive 
Euler characteristic. If p, q ^ 2 the isotropy subalgebra at I £ HP"" 1 is sp(p — 1) ffi 
Sp(q - 1) ffisp(l) acting on 

(U ffi R 3 ) ffi (MP- 1 ffi R 3 ) ffi (H 9_1 ffi R 3 ) ffi M ffi R 3 ) ffi M ffi U , 

where M = A4(p — 1, q — 1,H) and £7 is the adjoint representation of sp(l). Here 
<5 = 2 but hrk(Sp(p - 1) X Sp(l), HP -1 ffi R 3 ) = -8 . Thus the action has non-zero 
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homogeneity rank. 

Obviously this module appears in the slice even when q = 1, so we get no new 
vanishing homogeneity rank actions. 

4.7.5. Hi = p{K) with p irreducible representation of real type of the simple Lie 
group K . We here use again the dimensional condition (|4.ip that in this situation 
becomes 

dim K + rk K ^ 4 deg p - 8. 

Kollross in lemma 2.6 in [12] lists all the representations a of real type of Lie groups 
L such that 2dimL ^ degcr — 2. This condition is always looser than ours. Counting 
the dimensions for the groups and the representations from this list, we have that 
only the spin representation of K = Spin(7) and the standard representations of 
SO(n) satisfy the condition. The latter correspond to the case treated in subsection 
14.71 Let us compute hrk(Spin(7) x Sp(l), HP 7 ). As usually we consider the orbit 
through the quaternionic line I spanned by a pure tensor of K 8 © R 4 . It turns out 
to be the seven-dimensional sphere Spin(7)/G2 and the slice representation is the 
tensor product of the standard representation of G2 with the adjoint representation 
of Sp(l). It is well known (see e.g. [3 p. 11]) that this irreducible representation has 
trivial principal isotropy and from this follows that hrk(Spin(7) x Sp(l),HP 7 ) = 0. 

4.8. The subgroups of Sp(fc) x Sp(n — k). We analyse this case with the aid of 
the following lemma: 

Lemma 4.1. Let G C Sp(A^) be a compact Lie group acting with vanishing ho- 
mogeneity rank on HP^ -1 . Then G = G x Sp(n) acts with vanishing homogeneity 
rank on HP Ar +"- 1 = Ph^ © HP). 

Proof. If v is taken in HP, the G-orbit through [v] in EP N+n - 1 = P M (^ N © HP) 
is HP" -1 . Therefore the action of G has homogeneity rank zero if and only if the 
slice representation at this quaternionic orbit has vanishing homogeneity rank. Note 
that the last factor of the isotropy subgroup G x Sp(l) • Sp(n — 1) acts trivially on 
the slice £[„] ~ M. N . Consider now the natural projection of M. N \ {0} on HP W_1 . 
This is an equivariant fibration with fiber H. Thus arguing as in Proposition 12 .31 we 
deduce that 

hrk(G x Sp(l), U N ) = hrk(G, HP^ 1 ) + hrk(Sp(l), H) 

and the claim follows since both the homogeneity ranks in the right hand side of 
the equality vanish. □ 

As a consequence, combining the previous lemma with Proposition 12.31 we obtain 
the following 

Corollary 4.2. The group G C Sp(ra) acts on HP" -1 with vanishing homogeneity 
rank if and only if G x Sp(A^) C Sp(ra) x Sp(A^) on HP" +Ar - 1 does. 
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The previous corollary avoid the analysis of those subgroups of Sp(fc) x Sp(n — fc) of 
the form Hi x H2 where either Hi or H2 equals Sp(fc) or Sp(n — fc). Except for the 
diagonal action of Sp(fc)A when fc — n — k (which has already been excluded), it is 
therefore sufficient to analyse all the subgroups Hi x H2, where Hi C Sp(fc) acts 
on HP' 8-1 and H2 C Sp(n — k) acts on M¥ n ~ k ~ 1 both with vanishing homogeneity 
rank. 

The cases that we shall consider are given by all possible combinations of the 
following: 



Where p(Hi) (g> a and p{H 2 ) <S> a are orbit equivalent to isotropy representations of 
a quaternionic-Kahler symmetric space, where a is the standard representation of 



The case U(fc) x U(n — fc) has already been treated, the cases in which one of the 
factor is either Sp(fci) x Sp(fc2) or Sp(7i) x Sp^) give rise to vanishing homogeneity 
rank actions thanks to Lemma l4.ll 

The remaining cases can be all excluded with a common argument: we treat 
explicitly one of them and then we explain how to generalize. 
Consider for example G = E 7 x Spin(ll) acting on P H (H 28 H 16 ). Let 
E 7 /E 6 • U(l) C HP 27 C HP 43 be the maximal totally complex orbit of G. The 
factor U(l) x Spin(ll) of the isotropy acts on the second module of the slice 
C 27 © H 16 with non vanishing homogeneity rank, since it is neither the isotropy 
representation of a symmetric space of inner type nor it appears in the list of [7] . 
Observe now that all of the factors of the products Hi x H2 we are consid- 
ering admit a totally complex orbit (see [2])- All the cases can therefore be 
excluded in the same manner taking at a first step a maximal totally com- 
plex orbit for the group Hi, and then observing that the slice representation 
contains a module on which the isotropy acts with non vanishing homogeneity rank. 

The classification is now complete. In fact once one goes further the only possibility 
that can occur is the product of three factors Gi x G 2 x G 3 where all of Gi ^ Sp(rij) 
(otherwise this case can be treated with the aid of Lemma I4.1j) , where each Gi 
gives rise to vanishing homogeneity rank action on HP' li_1 . This case can be easily 
excluded applying Proposition ^. 31 to the product of two of the factors. 



H 2 



Hi 



U(fc), Sp(fci) x Sp(fc 2 ) with ki + k 2 = fc, 
SO(fc) ® Sp(l), Spin(7) ® Sp(l),p(if x ) 
U(n - fc),Sp(Zi) x Sp(Z 2 ) withZi + l 2 =n-k, 
SO(n - k) ® Sp(l), Spin(7) <g> Sp(l), p(H 2 ) 



Sp(l). 
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i) 


SO(fc) x SO(n-fc) 


1 < fe < n - 1 


ii) 


U(m) 


2m = n 


iii) 


SO(p) (8) SO(g) 


pq = n, 3 < p < q 


iv) 


Sp(p) Sp(q>) 


4pq = n 


v) 




H simple, p £ Irru, degp = n 


Table 2. Maximal subgroups of SU(n) 


i) 


SO(n) 




ii) 


Sp(m) 


2m = n 


iii) 


S(U(fc) x U(n - fc)) 


1 < k < n- 1 


iv) 


SU(p) (8) SU(g) 


p<7 = n, p > 3, g > 3 


v) 




simple, p £ Irrc, degp = n 


Table 3. Maximal subgroups of Sp(n) 


i) 


U(n) 




ii) 


Sp(fc) x 5p(n - jfe) 


1 > fe > n - 1 


iii) 


SO(p) ® Sp(g) 


pq = n, p > 3, q > 1 


iv) 


P(H) 


if simple, p e Irru, dcgp = 2n 



5. Appendix: Tables 
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